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Abstract: We construct two models of discrete-time replicator dynamics with time delay. In the 
social-type model, players imitate opponents taking into account average payoffs of games played 
some units of time ago. In the biological-type model, new players are born from parents who played in 
the past. We consider two-player games with two strategies and a unique mixed evolutionarily stable 
strategy. We show that in the first type of dynamics, it is asymptotically stable for small time delays 
and becomes unstable for big ones when the population oscillates around its stationary state. In the 
second type of dynamics, however, evolutionarily stable strategy is asymptotically stable for any size 
of a time delay. 
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stability of evolutionarily stable strategies in discrete replicator dynamics with time dela^ 

1. Introduction 

The evolution of populations can be often described within game-theoretic models. The pioneer of 
such approach, John Maynard Smith, introduced the fundamental notion of an evolutionarily stable 
strategy (1974, 1982). If everybody plays such a strategy, then the small number of mutants playing a 
different strategy is eliminated from the population. The dynamical interpretation of the evolutionarily 
stable strategy was later provided in (Taylor and Jonker, 1978; Hofbauer et al, 1979; Zeeman, 1981). A 
system of differential or difference equations, the so-called replicator equations, were proposed. They 
describe the time-evolution of frequencies of strategies. It is known that any evolutionarily stable 
strategy is an asymptotically stable stationary point of such dynamics (Hofbauer and Sigmund, 1988); 
Weibull, 1997). In fact, in two-player games with two strategies, a state of population is evolutionarily 
stable if and only if it is asymptotically stable. 

Recently Tao and Wang (1997) investigated the effect of a time delay on the stability of interior 
stationary points of the replicator dynamics. They considered a two-player game with two strategies 
and a unique asymptotically stable interior stationary point. They proposed a certain form of a time- 
delay differential replicator equation. They showed that the mixed evolutionarily stable strategy is 
stable if a time delay is small. For sufficiently large delays it becomes unstable. 

Here we discuss discrete-time replicator dynamics. We introduce a time delay in two different 
ways. In the so-called social model, we assume that players imitate a strategy with the higher average 
payoff, taking into account time-delayed information. When the limit of zero time interval is taken, 
one obtains equations discussed by Tao and Wang (1997). In the so-called biological model, we assume 
that the number of players born in a given time is proportional to payoffs received by their parents in 
a certain moment in the past. We analyze the stability of the interior stationary point in our discrete 
time-delay replicator dynamics. In the first model, we prove results analogous to those of Tao and 
Wang (1997). However, in our second model, we show the stability of the interior point for any value 
of the time delay. We prove our results in an elementary way, that is without any reference to the 
theory of time-delay equations. 

In Section 2, we present our discrete-time replicator dynamics with two types of time delay. In 
Section 3, we discuss a model with a social-type time delay and in Section 4, with a biological-type 
time delay. Discussion follows in Section 5. Proofs are given in Appendix A and B. 

2. REPLICATOR DYNAMICS 

We assume that our population is haploid, that is the offspring have identical phenotypic strategies 
as their parents (we assume that there are two different pure strategies; denote them by A and B). In 
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discrete moments of time, individuals compete in pairwise contests and the outcome is given by the 
following payoff matrix: 

A B 
A a b 

U = 

Bed, 

where the ij entry, i,j = A,B, is the payoff of the first (row) player when he plays the strategy 
i and the second (column) player plays the strategy j. We assume that both players are the same 
and hence payoffs of the column player are given by the matrix transposed to U; such games are 
called symmetric. From now on wc will assume that c > a and b > d so there exists a unique mixed 
evolutionarily stable strategy, x* = where 6i = c — a,52 = b ~ d, and 6 = di + 62- x* is the 
equilibrium fraction of an infinite population playing A. 

We will first discuss replicator dynamics without a time delay. We assume that during the small 
time interval e, only an e fraction of the population takes part in pairwise competitions, that is plays 
games. Let Pi{t), i = A,B, be the number of individuals playing at the time t the strategy A and 
B respectively, p{t) = PA{t) + PB{t) the total number of players and x{t) = a fraction of the 
population playing A. We write 

Pi{t + e) = {1 - e)m{t) + epi{t)Ui{ty, i = A,B, (2.1) 

where UA{t) = ax{t) + 6(1 — x{t)) and UB{t) = cx{t) + d(l — x{t)) are average payoffs of individuals 
playing A and B respectively. We assume that all payoffs are not smaller than hence pA and pB are 
always nonnegative and therefore < x < 1. 

The equation for the total number of players reads 

p{t + e) = (1 - e)p{t) + ep{t)U{t), (2.2) 

where U(t) = x{t)UA{t) + (1 — x{t))UB{t) is the average payoff in the population. When we divide 
(2.1) by (2.2) we obtain an equation for the frequency of the strategy A, 



(2.3) 
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Now we divide both sides of (2.3) by e, perform the limit e ^ 0, and obtain the well known 
differential replicator equation, 



^ = x{t)[UA{t)-U{t)]- (2.4) 
The above equation can also be written as 

^ = x{t){l - xmUAit) - Unit)] = -dx{t){l - x{t)){x{t) - X*). (2.5) 

It follows that X* is the globally (except the boundary points x = and x = 1) asymptotically 
stable stationary point of (2.5). 

In the following two sections we will study the stability of x* in the replicator dynamics with 

different forms of the time delay. 

3. SOCIAL-TYPE TIME DELAY 

Here we assume that individuals at time t replicate due to average payoffs obtained by their strategies 
at time t — r for some delay r > (see also a discussion after (3.6)). 
We propose the following equations: 



Pi{t + e) = (1 - e)pi{t) + epi{t)Ui{t - r); i = A, B. (3.1) 

Then for the total number of players we get 

p{t + e) = (1 - e)p{t) + ep{t)Uo{t - r), (3.2) 

where Uo{t - t) = x{t)UA{t - r) + (1 - x{t))UB{t - r). 

We divide (3.1) by (3.2) and obtain an equation for the frequency of the strategy A, 

and after some rearrangements we get 

xit + e) - x{t) = -ex{t){l - x{t))[x{t - r) - x*]^— (3.4) 
Now the corresponding replicator dynamics in the continuous time reads 

dx{t) 



dt 



x{t)[UA{t-T)-Uo{t-T)\ (3.5) 
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and can also be written as 



= x{t){l - x{t))[UA{t - r) - UB{t - r)] = -dx{t){l - x{t))ix{t - r) - x*). (3.6) 

The first equation in (3.6) can be also interpreted as follows. Assume that randomly chosen players 
imitate randomly chosen opponents. Then the probability that a player who played A would imitate 
the opponent who played B at time t is exactly x{t){l — x{t)). The intensity of imitation depends on 
the delayed information about the difference of corresponding payoffs at time t — r. We will therefore 
say that such models have a social-type time delay. 

Equations (3.5-3.6) are exactly the time-delay replicator dynamics proposed and analyzed by Tao 
and Wang (1997). They showed that if r < 5Tr/26id2, then the mixed evolutionarily stable strategy, 
X*, is asymptotically stable. When r increases beyond the bifurcation value 6^/26162, x* becomes 
unstable. We will show in Appendix A analogous results for our discrete equation (3.4). Our proofs 
will be elementary - we will not refer to the theory of time-delay equations. In particular, we will 
show that for any large enough time delay, r, x{t) oscillates around x*. 



4. Biological- TYPE time delay 

Here we assume that individuals born at time t — T are able to take part in contests when they become 
mature at time t or equivalently they are born r units of time after their parents played and received 
payoffs. We propose the following equations: 

p^{t + e) = (1 - e)pi{t) + epiit - T)Ui{t - r); i = A, B. (4.1) 
Then the equation for the total number of players reads 



p(t + .) = (1 - + .p(t)[M^t/.(* - r) + _ r)l (4.2) 

PA{t) PB{t) 

We divide (4.1) by (4.2) and obtain an equation for the frequency of the first strategy, 

x{t - T)UA{t - r) - x{t)U{t - t) 



x{t + e)-x{t)= e— ' ' ' r^^^ , (4.3) 

where U{t — t) = x[t — T)UA{t ~ t) + (1 — x[t — T))UB{'t ~ t)- We will prove in Appendix B that x* is 
asymptotically stable for any value of the time delay r. Here we will show our result in the following 
simple example. 
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The payoff matrix is given by C/ = ^ ^ ^ ^ hence x* = ^ is the mixed evolutionarily stable strategy 
which is asymptotically stable in the replicator dynamics without time delay. The equation (4.3) now 
reads 

.(t + .) - xW = - - r - ^-^W"" - - - . (4.4) 

(l-£)i;p)+2EJ^(S-T)(l-i(t-r)) 

After simple algebra we get 

1 1 ,s „ / XN x(t — t\(\ — xit — t\) , 

2 2 (i_e)-g(il^ + 2ex(t-r)(l-a;(t-r)) 

X 1 / / X Ix 1 

x{t + e)-- = {x{t) - - 



2 ^ I + ^^2x{t - r){l - x{t - r)) 



hence 



Wi + e)-^|< (4.6) 



It follows that X* is globally asymptotically stable. 



5. Discussion 



We proposed and analyzed certain forms of discrete-time replicator dynamics with a time delay. We 
introduced the time delay on the level of the number of individuals playing different strategies and 
not on the level of their relative frequencies. We showed that the stability of a mixed evolutionarily 
stable strategy depends on the particular form of the time delay. 

It would be interesting to analyze time-delay replicator models taking into account the limited 
capacity of an environment (for example in the form of a logistic equation) and stochastic effects 
resulting from mutations and a random character of interactions. In the latter case, individuals 
play with particular opponents and not against the average strategy like in the standard replicator 
dynamics. 
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Appendix A. 

Theorem 1. x* is asymptotically stable in the dynamics (3.4) if t is sufficiently small and unstable 
for large enough r. 
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Proof: We will assume that r is a multiple of e, r = me for some natural number m. Observe first 
that if x{t — r) < x*, then x{t + e) > x{t), and if x{t — r) > x*, then x{t + e) < x{t). Let us assume 
first that there is t' such that x{t'),x{t' — e),x{t' — 2e), ...,x{t' — t) < x*. Then x{t), t>t' increases 
up to the moment ti for which x{ti — r) > x* . If such ti does not exist then x{t) ^t^oo x* and the 
theorem is proved. Now we have x* < x{ti — r) < x{ti — r + e) < ... < x[ti) and x{ti + e) < x[ti) 
so ti is a turning point. Now x{f?) decreases up to the moment t2 for which x{t2 — t) < x* . Again, 
if such t2 does not exist, then the theorem follows. Therefore let us assume that there is an infinite 
sequence, ti, of such turning points. Let r]i = \x{ti) — x*\. We will show that rji — >j-+oo 0- 
For t G {ti, ti + e,... , tj+i — 1} we have the following bound for x{t + e) — x{t): 

+ " -"">l<i'" l-. + l(t-r) - <^-') 

This means that 



7]i+i <{m + l)eK7]i, (A.2) 
where K is the maximal possible value of (t-r)) ' S^t that if 

r<-i-e, (A.3) 

then r]i — >-i^oo so x{t) converges to x*. 

Now if for every t, \x{t + e) — x*\ < maXfcg|o,i,...,m} l^it — ke) — x*|, then x{t) converges to x* . 
Therefore assume that there is t" such that \x{t" + e) — x*\ > maXfcg|o,i,...,m} l^it" — ke) — x*\. If r 
satisfies (A.3), then it follows that x{t + e), ...,x{t + e + r) are all on the same side of x* and the 
first part of the proof can be applied. We showed that x{t) converges to x* for any initial conditions 
different from and 1 hence x* is globally asymptotically stable. 

Now we will show that x* is unstable for any large enough r. 

Let 7 > be arbitrarily small and consider a following perturbation of the stationary point x*: 
x{t) = x*,t < and x{e) = x* + 7. It folows from (3.4) that x{ke) = x{e) for A; = 1, ...,m + 1. Let 
K' = mina,g[^*_^ ,2,.+^] . If yeK'7 > 27, that is r > then it follows from (3.4) that 

after m/2 steps (we assume without loss of generality that m is even) x((m + 1 + m/2)e) < x* — 7. 
In fact we have x((2m + l)e) < . . . < x{{m + l)e) and at least m/2 of x' s in this sequence are 
smaller than x* — 7. Let i > {2m + l)e be the smallest t such that x{t) > x* — 7. Then we have 
x{t — me), . . . ,x{t — e) < X* — 7 < x{t) hence after m/2 steps, x{t) crosses x* + 7 and the situation 
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repeats itself. We showed that if 

then there exists an infinite sequence, tj, such that \x{ti) — x*\ > 7 and therefore x* is unstable. 
Moreover, x{t) oscillates around x*. 

Appendix B. 

Theorem 2. x* is asymptotically stable for any r in the dynamics (4-3). 

Proof: Let ct = ^^7^^- Observe that if x{t) < x*, then a > x{t), if x{t) > x*, then c* < x{t), and 
if x{t) = X*, then ct = x*. We can write (4.3) as 

xit + e) - xit) = ,^(^ - rP.jt - r) - Ct-rUjt^) + ^^-.^(t - r) - - r) 

and after some rearrangements we obtain 



i^-^)M^+^U{t-r) 



X{t + 6) - Ct-r = {X{t) - ct-r)--^^^^^ -. (B.2) 

We get that at time t + e,xis closer to Ct-r than at time t and it is on the same side of ct-r- We 
will show that c is an increasing or a constant function of x. Let us calculate the derivative of c with 
respect to x. 



fix 

{xUa + (1 - x)Ub 

where 



/(x) = (ac + hd- 2ad)x^ + 2d{a - b)x + bd. (B.4) 

A simple analysis shows that / > on (0, 1) or / = on (0, 1) (in the case of a = d = 0). Hence 
c{x) is either an increasing or a constant function of x. In the latter case, Va;c(a;) = x*, as it happens 
in our example in Section 4, and the theorem follows. 

We will now show that 

\x{t + T + e)-x*\ < max{|x(t) - x*\, \x{t + r) - x*\} (B.5) 

hence x{t) converges to x* for any initial conditions different from and 1 so x* is globally asymptot- 
ically stable. 

If x{t) < X* and x{t + r) < x*, then x{t) < ct < x* and also x{t + r) < ct+r < x*. From (B.2) we 
obtain 
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x{t + T) < x{t + T + e) < Ct if X{t + T) < Ct 
X (t) < X {t + T + e) = ct if x{t + T)=ct 
x{t) <ct < x{t + T + e) <x{t + T) if X (i + r) > Ct 

hence (B.5) holds. 

If x (i) > X* and x{t + t) < x*, then x {t + t) < x* < Ct < x (t) and either x{t + t) < x{t + T + e) < 
X* or X* < X {t + T + e) < Ct which means that (B.5) holds. 

The cases of x (t) > x*, x {t + t) > x* and x {t) < x* , x {t + t) < x* can be treated analogously. 
We showed that (B.5) holds. 

References 

Hofbauer, J., Shuster, P. Sigmund, K., 1979. A note on evolutionarily stable strategies and game 
dynamics. J. Theor. Biol. 81, 609-612. 

Hofbauer, J., Sigmund, K., 1988. The Theory of Evolution and Dynamical Systems. Cambridge 
University Press, Cambridge. 

Maynard Smith, J., 1974. The theory of games and the evolution of animal conflicts. J. Theor. Biol. 
47, 209-221. 

Maynard Smith, J., 1982. Evolution and the Theory of Games. Cambridge University Press, Cam- 
bridge. 

Tao, Y. and Wang, Z., 1997. Effect of time delay and evolutionarily stable strategy. J. Theor. Biol. 
187, 111-116. 

Taylor, P. D., Jonker, L. B., 1978. Evolutionarily stable strategy and game dynamics. Math. Biosci. 
40, 145-156. 

Weibull, J., 1997. Evolutionary Game Theory. MIT Press, Cambridge. 

Zeeman, E., 1981. Dynamics of the evolution of animal conflicts. J. Theor. Biol. 89, 249-270. 



